Reward-Weighted Regression Converges to a Global Optimum

Reward-Weighted Regression (RWR) belongs to a family of widely known iterative Reinforcement Learning algorithms based on the Expectation-Maximization framework. In this family, learning at each iteration consists sampling batch trajectories using current policy and fitting new maximize return-weighted log-likelihood actions. Although RWR is yield monotonic improvement under certain circumstances, whether which conditions converges optimal have remained open questions. paper, we provide for first time proof that global optimum when no function approximation used, in general compact setting. Furthermore, simpler case with finite state action spaces prove R-linear convergence state-value optimum.